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SPECIFICATION 

Three-Dimensional Image Processing Apparatus 

FIELD OF THE INVENTION 

This invention relates to a three-dimensional image processing 
apparatus and an external memory device to be used therewith. More 
particularly, the invention relates to a three-dimensional image 
processing apparatus which displays on a display device an image of 
a player controlled object or other objects, existing in three- 
dimensional space, from the perspective of a predetermined "camera" 
position (the point of view). 

BACKGROUND AND SUMMARY OF THE INVENTION 

The conventional so-called 3D (3 -Dimensional) video game 
uses player controlled or operable objects (objects operable by an 
operator) configured with three-dimensional data when viewed by an 
apparent camera at predetermined angles and distances, thereby 
obtaining displayed images. In the conventional game, however, if a 
background image (e.g., a wall) or an object used as an opponent 
character (another object) comes between the player controlled object 
and the "camera", or if another object is moved to interrupt the line of 
sight between the operable object and the camera, the operable object 



can not be viewed in the three-dimensional world. To this end, there 
has been a limitation in the conventional 3D games in that the other 
object has to be arranged by a program not to exist between the 
operable object and the camera. 

It is therefore an object of the present invention to provide an 
image processing apparatus which is capable of displaying an 
operable object at substantially all times and hence free from 
limitation in arranging other objects. 

The illustrative image processing apparatus displays on a 
display an operable object image and another object existing in a 
three-dimensional space from a predetermined viewing or 
"photographic" position. The image processing apparatus includes an 
external memory which stores operable object and the other object 
data and a predetermined program. The system uses an input 
controller which inputs data which alters the position of the operable 
object in the three-dimensional space. Operable object position data 
generating hardware and software generates operable object position 
data so as to alter the position of the operable object in the three- 
dimensional space based on the data input by the input controller. 
The three-dimensional data is created based on the data stored in the 
external memory and the operable object position data. Point of view 
position data is generated representing photographing position data in 



the three-dimensional space for displaying the operable object. The 
system detects whether or not the other object exists between the 
"camera" view position and the operable object position. If so, the 
system alters the photographing position data such that the other 
object is not existent between the photographing position and the 
operable object position when the detecting means detects existence 
of the other obj ect. The system creates display data for displaying 
the image of the operable object photographed from a predetermined 
position in the three-dimensional space based on the three- 
dimensional data and the photographing position data; and image 
signal generating circuitry outputs an image signal to the display 
based on the generated display data. 

The system determines whether or not there is a possibility of a 
collision between the operable object and a polygon plane of the 
other object. If there is a possibility of a collision of the operable 
object with the other object, the camera position is changed so that 
the other object does not exist between the operable object and the 
camera. Therefore, the operable object is "photographed" without 
interference by the other object. 

In accordance with the present invention, even if another 
object is permitted to freely move, it is possible to display at 
substantially all times an operable object on a screen of a display. 
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Consequently, if the present invention is applied to a game apparatus, 
the operable object can be displayed at all times on a display, even 
for a game that involves an operable object and a number of other 
objects moving around on the display screen. 

The above and other objects, features, aspects, and advantages 
of the present invention will become more apparent from the ensuing 
detailed description of the present invention when taken in 
conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is an illustrative schematic external view showing one 
embodiment of an exemplary image processing system; 

Figure 2 is an exemplary block diagram of an image 
processing apparatus in the Figure 1 embodiment; 

Figure 3 is an illustrative view showing a CPU memory map 
for use in the Figure 2 embodiment, showing an external memory and 
a W-RAM address space; 

Figure 4 is a block diagram showing an exemplary controller 
control circuit in the Figure 2 embodiment; 



Figure 5 is an illustrative view for explaining a 
modulating/demodulating method; 

Figure 6 is an illustrative view showing a memory map of a 
RAM in Figure 4; 

Figure 7 is a perspective view of a controller of the Figure 2 
embodiment as viewed from the top; 

Figure 8 is a perspective view of the controller of the Figure 2 
embodiment as viewed from the bottom; 

Figure 9 is a block diagram showing in detail the controller and 
an expansion device; 

Figure 10 shows illustrative data from a controller's analog 
joystick of the respective keys/buttons; 

Figure 1 1 shows illustrative transmission and reception data 
when a command "0" is transmitted from the controller control 
circuit; 

Figure 12 shows illustrative transmission and reception data 
when a command "1" is transmitted from the controller control 
circuit; 



Figure 13 shows illustrative view of transmission and reception 
data when a command "2" is transmitted from the controller control 
circuit; 

Figure 14 shows illustrative view of transmission and reception 
data when a command "3" is transmitted from the controller control 
circuit; 

Figure 15 is a flowchart showing operation of the CPU of the 
Figure 2 embodiment; 

Figure 16 is a flowchart showing operation of the bus control 
circuit of the Figure 2 embodiment; 

Figure 17 is a flowchart showing operation of the controller 
control circuit of the Figure 2 embodiment; 

Figure 18 is a flowchart showing operation of the controller 
circuit of the Figure 2 embodiment; 

Figure 19 shows illustrative transmission and reception data 
when a command "255" is transmitted from the controller control 
circuit; 

Figure 20 is an illustrative view showing a state when a wall 
exists between the operable object (Mario) and the camera; 



Figure 21 is an illustrative view showing point of view 
movement regarding Figure 20; 

Figure 22 is a flowchart showing operation for a camera 
ftirning-around process; 

Figure 23 is a flowchart showing a collision-determining 
routine; 

Figure 24 is an illustrative view showing a wall polygon; 

Figure 25 is an illustrative view showing each polygon; 

Figure 26 is an illustrative view showing a projected surface; 

Figure 27 is an illustrative view showing a state that a 
projection is onto a YX plane; 

Figure 28 is an illustrative view showing a state that a 
projection is onto an XY plane; and 

Figure 29 is an illustrative view showing a normal vector of the 
plane and a point of view vector of the camera. 
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EMBODIMENTS 

Figure 1 depicts an exemplary image processing system 
according to one embodiment of the present invention. The image 
processing system is for example a video game system, which 
5 comprises an image processing apparatus 10, a ROM cartridge 20 (as 
one example of an external memory device), a display 30 (as one 
example of a display means) connected to the image processing 
apparatus main body 10, a controller 40 as one example of a player 
controller or operating device. The preferred controller is shown in 

10 Figures 7 and 8 and is described below. A RAM cartridge 50 is one 
example of an extension device detachably attached to the controller 
40. The external memory device stores image data and program data 
for image processing for games, and audio data for music, sound 
effects, etc. A CD-ROM or a magnetic disc may alternatively be 

15 employed in place of the ROM cartridge. Where the image 

processing system of this example is applied to a personal computer, 
an input device such as a keyboard or a mouse is used as the player 
operating device. 

Figure 2 is a block diagram of the image processing system of 
20 this example. The image processing apparatus 10 incorporates 
therein a central processor unit (hereinafter "CPU") 1 1 and a bus 
control processing circuit 12. The bus control circuit 12 is connected 
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to a cartridge connector 13 for detachably attaching the ROM 
cartridge 20, as well as a working RAM 14. The bus control 
processing circuit 12 is connect to an audio signal generating circuit 
15 for outputting an audio signal process by the CPU 1 1 and a video 
5 signal generating circuit 16 for outputting a video signal, and further 
with a controller control circuit 17 for serially transferring operating 
data of one or a plurality of controller(s) 40 and/or data from RAM 
cartridge(s) 50. The controller control circuit 17 is connected with 
controller connectors (hereinafter abbreviated as "connectors") 181- 

10 1 84 which are provided at a front face of the image processing 
apparatus 10. To the connector 18 is detachably connected a 
connection jack 41 and the controller 40 through a cable 42. Thus, 
the connection of the controller to the connector 181-184 places the 
controller 40 into electric connection to the image processing 

15 apparatus 10, enabling transmission and reception of data 
therebetween. 

More specifically, the bus control processing circuit 12 inputs 
commands output as parallel signals from CPU 1 1 via a bus, 
performs parallel to serial conversion, outputs command as serial 
20 signals to the controller control circuit 17, and converts serial signal 
data input from the controller control circuit 17 into parallel signals 
and outputs such signals to the bus. The data outputted through the 
bus is subject to processing by CPU 1 1, or is stored in W-RAM 14. 
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The W-RAM 14 is a memory temporarily storing data to be 
processed by CPU 11, wherein read-out and write-in of data is 
possible through the bus control circuit 12. 

Figure 3 is a diagrammatic illustration showing memory 
regions assigned to respective memory spaces. The memory spaces 
accessible by the CPU via the bus control processing circuit 12 
involves an external memory address space of the ROM cartridge 20 
and a memory address space of the W-RAM 14. The ROM cartridge 
20 is structured by mounting on a board a ROM stored with data for 
game processing and accommodating the same board in a housing. 
The ROM includes an image data storage region 201 for storing 
image data required to cause the image processing apparatus 10 to 
generate image signals for the game, and a program data region 202 
for storing program data required for predetermined operation of the 
CPU 1 1. In the program data region 202, there are stored an image 
display program for performing image display processing based on 
image data 201, a time-measuring program for carrying out 
measurement of time, and a determination program for determining 
that the cartridge 20 and an extension or expansion device 50, are in a 
predetermined relationship. The details of the time-measuring 
program and the determination programs are described below. The 
memory region of W-RAM 14 includes a region 141 for temporarily 
storing data representative of an operating state from a control panel. 
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Figure 4 is a more detailed circuit diagram of a controller 
control circuit 17. The controller control circuit 17 transmits and 
receives data in serial form to and from the bus control processing 
circuit 12 and the controller connectors 181-184, and includes a data 

5 transfer control circuit 1 7 1 , a signal transmitting circuit 1 72, a signal 
receiving circuit 173 and a RAM 174 for temporarily storing 
transmission and reception data. The data transfer control circuit 171 
includes a parallel-serial conversion circuit and a serial-parallel 
conversion circuit for conversion of data format during data transfer, 

10 and also performs control of write-in and read-out of the RAM 174. 
The above-mentioned serial-parallel conversion circuit converts 
serial data supplied from the bus control processing circuit 12 into 
parallel data to provide such data to the RAM 174 or the signal 
transmitting circuit 172. The parallel-serial conversion circuit 

15 converts parallel data supplied from the RAM 1 74 or the signal 
receiving circuit 173 into serial data to provide such data to the bus 
control processing circuit 12. The signal transmission circuit 172 
converts parallel data for signal read-in control of the controller 40 
supplied from the data transfer control circuit 171 and write-in data 

20 (parallel data) to the RAM cartridge 50 into serial data, which serial 
data is transmitted through a corresponding channel CHI - CH4 to 
each of the plurality of controllers 40. The signal receiving circuit 
173 receives serial read-out data, representative of an operating state 
of each of the controller 40, input through a corresponding channel 
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CHI - CH4 to each of the controller 40 as well as read-out data from 
the RAM cartridge 50, to convert such data into parallel data to 
provide it to the data transfer control circuit 171. 

The signal transmitting circuit 172 and the signal receiving 
circuit 173 adopt a duty-cycle modulation and demodulation 
(hereinafter referred to as "modulation/demodulation") method as one 
example of the modulation/demodulation method that may be 
employed here. The duty-cycle modulation/demodulation method, as 
shown in Figure 5, is a modulation/demodulation method wherein 
" 1 " and "0" are represented by varying a Hi time period and a Lo 
time period for a signal at a certain interval. Explaining the 
modulation/demodulation method in more detail, when data to be 
transmitted in serial is a logical "1", a signal having, within one cycle 
period T, a high-level period tH rendered longer than a low-level 
period tL (tH > tL) is transmitted, while when data to be transmitted 
is a logical "0", a signal having, within one cycle period T, tH 
rendered shorter than tL (tH < tL) is transmitted. 

The demodulation method samples on a serial signal received 
(bit transmission signal) so as to monitor at all times whether the 
received signal is at a high level or a low level, wherein one cycle is 
expressed as T = tL + tH provided that time period of low till change 
to high is tL and time period of high till change to low is tH. In this 
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case, the relationship of tL and tH being tL < tH is recognized as 
logical "1", while tL > tH is recognized as logical "0", thereby 
achieving demodulation. If a duty-cycle modulation/demodulation 
method like this is employed, there is no necessity of transmitting 
5 data in synchronism with a clock signal, offering an advantage that 
transmission and reception of data are available with only one signal 
!"!! line. If two signal lines are available, another 

"i modulation/demodulation method may be utilized. 

isjir: 

I'Si 

\\ The RAM 1 74 includes memory regions or memory areas 1 74a 

m 

i* io - 174H as shown in a memory map of Figure 6. Specifically, the 

a 

area 174a stores a command for channel 1, while the area 174b stores 

Li, 

H' transmission data and reception data for channel 1. The 174c stores a 

O 

ffi command for channel 2, while the area 174d stores transmission data 

and reception data for channel 2. The area 174e stores a command 
15 for channel 3, while the area 1 74f stores transmission data and 
reception data for channel 3. The area 174g stores a command for 
channel 4, while the area 174h stores transmission data and reception 
data for channel 4. 

Accordingly, the data transfer control circuit 1 7 1 operates to 
20 write-in control to the RAM 174 data transferred from the bus control 
processing circuit 12 or operating state data of the controller 40 
received by the signal receiving circuit 173 or read-out data from the 
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RAM cartridge 50, and read data out of the RAM 174 based on a 
command from the bus control processing circuit 12 to transfer it to 
the bus control circuit 12. 

Figure 7 and Figure 8 are external views of the top and rear 
surfaces of the presently preferred controller 40. The controller 40 is 
in a shape that can be grasped by both hands or one hand, and has a 
housing having an exterior formed projecting with a plurality of 
buttons, which when depressed, are operable to generate an electrical 
signal and a vertically-standing control portion. Specifically, the 
controller 40 is constituted by an upper housing and a lower housing. 
As shown in Figure 7, the controller 40 housing has an operating area 
formed on an upper surface in a planar shape running from switch 
403 through buttons 404. In the operating area of the controller 40, 
there are provided a cross-shaped digital direction switch (hereinafter 
referred to as "cross switch") 403 on a left side, a plurality of button 
switches (hereinafter merely abbreviated as "switches") 404A - 404F 
on a right side, a start switch 405 generally at a laterally central 
portion, and a joystick 45 for allowing analog input at a centrally . 
lower portion. The cross switch 403 is a direction switch for 
designating the direction of movement of a player controlled heroic 
character or a cursor, which has upper, lower, left and right 
depression points to be used for designating movement in four 
directions. The switches 404A - 404F, being different by game 
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software, are used, for example, to launch a missile in a shooting 
game, or to designate various actions such as jumping, kicking, or 
taking a matter in an action game. Although the joystick 45 is used 
in place of the cross switch 403 to designate the direction of 
5 movement of a player controlled heroic character or the like, it can 
designate direction at the entire angular range over 360 degrees, 
q being utilized as an analog direction designating switch. 

n 

a:: 
sjjais 

fU The controller 40 housing has three grips 402L, 402C and 

jss:: 

y 402R formed in a manner projecting downward from three locations 

1 10 of the operating area. The grips 402L, 402C and 402R are in such 
!*"!' rod-shapes that, when seized by the hand, they are contoured by the 

j;°J palm, the middle finger, the finger between the middle and little 

ft' fingers and the little finger. Each grip is formed a little thin at a base 

portion, thick at an intermediate portion and thinner toward an open 
15 end (downward in Figure 7). The lower housing of the controller 40 
has an insertion aperture 409 formed at a centrally upper portion 
which projects from the underside for detachabiy attaching a RAM 
cartridge 50 as an expansion device. The housing has a button switch 
406L and a button 406R provided left and right on upper side faces 
20 thereof at locations corresponding to the positions to which the left 
and right index fingers of a player extend. In a back surface at the 
base portion of the central grip 402C, a switch 407 is provided as a 
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switch having a function alternative to the switch 406L when the 
joystick 45 is used in place of the cross switch 403. 

The lower half of the housing on a back surface side extends 
toward a botton surface to have an aperture 408 formed at a tip end 

5 thereof. At a deep end of the aperture 408, a connector (not shown) 
to which an extension cartridge 50 is to be connected is provided. In 
the aperture 408 is also formed a lever 409 for ejecting the cartridge 
50 inserted in the aperture 408. On a side opposite to the lever 409 in 
the aperture 408 for insertion of an extension cartridge 50, a cut-out 

10 410 is formed, which cut-out 410 provides a space for pulling out the 
extension cartridge 50 upon taking out the extension cartridge 50 by 
using the lever 409. 

Figure 9 is a detailed circuit diagram of a controller 40 and a 
RAM cartridge 50 (as one example of an extension device). The 

15 controller 40 incorporates within its housing, electronic circuits such 
as an operation signal processing circuit 44, etc. in order to detect 
operating states of the switches 403 - 407 or the joystick 45 or the 
like and transfer detected data to the controller control circuit 17. 
The operation signal processing circuit 44 includes a signal receiving 

20 circuit 44 1 , a control circuit 442, a switch signal detecting circuit 
443, a counter circuit 444, a signal transmitting circuit 445, a joyport 
control circuit 446, a reset circuit 447 and a NOR gate 448. 
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The signal receiving circuit 441 converts a serial signal, such 
as a control signal transmitted from the controller control circuit 17, 
write-in data to the RAM cartridge 50, etc., into a parallel signal to 
supply it to the control circuit 442. The control circuit 442 generates 
5 a reset signal to cause resetting (e.g., setting to 0) on measured values 
of an X-axis counter 444X and a Y-axis counter 444Y included in the 

O counter 444, when the control signal transmitted from the controller 

Q 

£ control circuit 1 7 is a reset sianal for an X, Y coordinate of the 

fU 

joystick 45. The jovstick 45 includes photo- interrupters for the X- 

M 

LFI io axis and Y-axis so as to generate the number of pulses proportional to 

0 the amount of inclination of a lever in directions of X-axis and Y- 

H> axis, providing respective pulse signals to the counters 444X and 

O 444Y. The counter 444X. when the joystick 45 is inclined in the X- 

W 

axis direction, measures the number of pulses generated in proportion 
15 to the amount of inclination. The counter 444Y measures the number 
of pulses generated in proportion to the amount of inclination, when 
the joystick 45 is inclines in the Y-axis direction. Accordingly, the 
resultant vector, determined by the measured values in X-axis and Y- 
axis of the counter 444X and the 444 Y, determines the direction of 
20 movement and the coordinate position for the heroic character or the 
cursor. The counter 444X and the counter 444Y are also reset of 
their measured values by a reset signal supplied from the reset signal 
generating circuit 447 upon turning on the power supply, or a reset 
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signal supplied from the switch signal detecting circuit 443 when the 
player depresses simultaneously two switches previously determined. 

The switch signal detecting circuit 443 responds to an output 
command signal representing a switch state supplied at a constant 
period (e.g., at a 1/30-second interval as a frame period of a 
television), and reads a signal that is varied by the state of depression 
of the cross switch 403 and the switches 404A - 404F, 405, 406L, 
406R and 407 to supply it to the control circuit 442. 

The control circuit 442 responds to a read-out command signal 
of operating state data from the controller control circuit 17, and 
supplies the operating state data of the switches 403-407 and the 
measuring values of the counters 444X, 444Y to the signal 
transmitting circuit 445 in a predetermined data-format order. The 
signal transmitting circuit 445 converts these parallel signals output 
from the control circuit 442 into serial data to transfer them to the 
controller control circuit 17 via a conversion circuit 43 and a signal 
line 42. . 

The control circuit 442 is connected to an address bus, a data 
bus, and a port control circuit 446 through a port connector. The port 
control circuit 446 performs input-output control (or signal 
transmission or reception control) on data according to commands by 
the CPU 11, when the RAM cartridge 50 (as one example of an 
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extension device) is connected to a port connector 46. The RAM 
cartridge 50 includes a RAM 5 1 and a timer chip 53 as one example 
of a time-related information generating means (or a calendar timer) 
connected to the address bus and the data bus, a battery 52 connected 
5 thereto for supplying power source to the RAM 5 1 and the timer 
counter 53, and a decoder 54 for activating the timer counter 53 when 
a predetermined address is given. The RAM 5 1 is a RAM that has a 
capacity lower than a half of a maximum memory capacity accessible 
by using an address bus, and is comprised for example of a 256 k-bit 

10 RAM. This is because of avoiding duplication between the write- 
in/read-out address of the RAM and the read-out address of the timer 
chip 53 by reading out a value of an arbitrary counter within the timer 
chip 53 when the highest order bit becomes " 1 ". The RAM 5 1 stores 
backup data associated with a game, so that, if the RAM cartridge 50 

15 is removed out of the port connector 46, the stored data is kept by 
receiving power supplied from the battery 52. The details of the kind 
of data stored by the RAM 51, writing data therein, and utilization of 
the data stored is described below. 

Figure 10 is a graphical illustration of a data format by which 
20 the image processing apparatus reads out data representative of an 
operating state of switches 403 - 407 and joystick 45 from the 
controller 40. The data generated by the controller 40 is configured 
as 4-byte data. The first-byte represents B, A, G, START, upper, 
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lower, left and right, i.e. represents the depression of the switch 
404B, 404A, 407, 405 and the four cross switch 403 directions. For 
example, when the button B, i.e., the switch 404B, is depressed, the 
highest order bit of the first byte becomes "1". Similarly, the second- 
byte represents JSRST, 0 (not employed in the embodiment), L, R, E, 
D, C, and F, i.e., the depression of the switch 409, 406L, 406R, 404E, 
404D, 404C and 404F. The third byte represents by binary digit the 
X coordinate value (the value measured by the X counter 444X) 
which value is dependent upon the inclination angle of the joystick 
45 in the X direction. The fourth byte represents by binary digit the 
Y coordinate value (the value measured by the Y counter 444Y) 
which value is dependent upon the inclination angle of the joystick 
45 in the Y direction. Because the X and Y coordinate values are 
expressed by 8 bits of binary digits, the conversion into decimal 
digits makes it possible to represent the inclination of the joystick 45 
by a numeral from 0-255. If the highest order bit is expressed by a 
signature denoting a negative value, the inclination angle of the 
joystick 45 can be expressed by a numeral between -128 and 127. 

Referring to Figure 1 1 to Figure 14, an explanation will be 
made on a format for the signals transmitted and received between 
the image processing apparatus 10 and the controller 40. 
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Figure 1 1 is an illustrative representation of a format for the 
signals transmitted and received between the image processing 
apparatus 10 and the controller 40 for identification of the type of a 
controller 40 by the image processing apparatus 10. The image 
5 processing apparatus 10 transmits a type data request signal of a 
command "0" configured by 1 byte (8 bits) to the control circuit 442 
within the controller 40, and receives in response thereto, 3 bytes of a 
type data signal, concerning the controller 40, of TYPE L (1 byte), 
TYPE H (1 byte) and the status generated by the control circuit 442. 

10 Here, TYPE L and TYPE H are data representative of a function of a 
device or apparatus connected to connector 46. The respective data 
of TYPE L and TYPE H are data inherent to the type of a RAM 
cartridge 50. Based on the data, the image processing apparatus 10 
identifies the type of a controller 40, i.e., the type of a RAM cartridge 

15 50 being connected to the controller 40. The type of RAM cartridge 
50 involves for example a type merely mounted with a RAM 5 1, a 
type mounted with a RAM 5 1 together with a timer chip, and a type 
mounted with a RAM 5 1 together with a liquid crystal display. In the 
present embodiment, the type mounted with a RAM 5 1 and a time 

20 chip is explained in detail. Meanwhile, the status data is data that 
represents whether or not the port is connected to an extension device 
such as a RAM cartridge 50 and whether or not an extension device 
has been connected thereto after resetting. 
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Figure 12 is an illustrative representation of a format for the 
signal transmitted and received between the image processing 
apparatus 10 and the controller 40 for determining the operating state 
of the controller 40 by the image processing apparatus 10. The image 
5 processing apparatus 10 transmits a controller data request signal of a 
command "1" configured by 1 byte (8 bits) to the control circuit 442 
within the controller 40, and receives in response thereto an operating 
state data signal, concerning the controller 40, generated by the 
control circuit 442. Based on the operating state data, the image 
10 processing apparatus 10 acknowledges how the operator operates the 
controller 40 for utilization for varying the image. The operating 
state data signal has been described in detail in the explanation on 
Figure 10. 

Figure 13 is an illustrative representation of a format for a read 
15 data signal when the image processing apparatus 10 reads data from 
the RAM 5 1 within the RAM cartridge 50 which is connected to 
controller 40. The image processing apparatus 10 transmits to 
control circuit 442, a read command signal of a command "2" 
configured by 1 byte (8 bits), an address H (8 bits) signal 
20 representative of the higher order bits of an address, an address L (8 
bits) signal representative of the lower order bits of an address and an 
address CRC (5 bits) signal for checking for transmission errors of 
address data of the address H signal and address L signal. The image 
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processing apparatus receives in response thereto a storage data 
signal, for the RAM 51, generated by the control circuit 442 and a 
data CRC (8 bits) signal for checking for data transmission error. 
Incidentally, to read out time-related information of the timer chip 53 
5 by the image processing apparatus 1 0, it is satisfactory to read out 
addresses of 8000h or longer by merely rendering the address H 
H' signal value greater than 8 Oh. 

,«»:, 
.sa:. 

w 

;f: Figure 14 is an illustrative representation of a format for a 

IT '! write data signal when the image processing apparatus 1 0 writes data 

If 1 10 into the RAM 5 1 within the RAM cartridge 50 connected to 

h; controller 40. The image processing apparatus 10 transmits, to the ^ _ 

[™ control circuit 442, a write command signal of a command "3" 

° configured by 1 byte (8 bits), an address H (8 bits) signal 

representative of a higher order bit of an address, an address L signal 
15 and an address H signal representative of a lower order bit (3 bits) of 
an address, an address CRC (5 bits) signal for checking for 
transmission error of address data of the address L signal, and a 32- 
byte write-in data signal to be written into the RAM 5 1 . The image 
processing apparatus 10 receives in response thereto a data CRC (8 
20 bits) signal generated by the control circuit 442 for checking for data 
reception error. The image processing apparatus 10 receives the 
CRC signal to perform CRC checking with the transmitted write-in 
data, and judges based thereon that the data has correctly been 
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written into the RAM 51. Incidentally, to reset for example date and 
time by writing time-related information into the timer chip from the 
image processing apparatus 10, it is satisfactory to perform writing 
into addresses of 8000h or higher by merely rendering the address H 
signal value greater than 80h. 

The operation of data transmission and reception between the 
image processing apparatus 10 and the controller 40 will now be 
explained. 

Referring first to a flowchart for the CPU of the image 
processing apparatus 10 in Figure 15, explanations will be made on 
image processing. At a step SI 1, CPU 1 1 is initialized based on an 
initial value (not shown) stored in the program data area 202 in 
Figure 3. Then, at a step S 12, the CPU 1 1 outputs a control pad data 
request command stored in the program data area 202 to the bus 
control circuit 12. At a step S 13, the CPU 1 1 carries the desired 
image processing based on the program stored in the program data 
area 202 and the image data area 201. While the CPU 1 1 is 
executing step S 13, the bus control processing circuit 12 is executing 
steps S21 - S24 of Figure 16. Then, at a step SI 4, the CPU 1 1 
outputs image data based on the control pad data stored in the control 
pad data area 141 in Figure 3. After completing step SI 4, the CPU 
branches to steps S 12 and repeats the execution of steps S12 - SI 4. 
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The operation of the bus control processing circuit 12 is 
explained in conjunction with Figure 16. At a step S21, the bus 
control circuit 12 determines whether or not the CPU 1 1 has output a 
controller data request command (a request command for data 
5 relating to the switches of the controller 40 or data on the extension 
device 50). If a controller data request command has been output, the 
process proceeds to a step S22. At the step S22, the bus control 
circuit 12 outputs a command for reading in data of the controller 40 
(command 1 or command 2 referred above) to the controller control 

10 circuit 17. Then, at a step S23, the bus control circuit 12 determines 
whether or not the controller control circuit 17 has received data from 
the controller 40 to store in the RAM 174. If the controller control 
circuit 17 has not received data from the controller 40 to store in the 
RAM 174, the bus control circuit 17 waits at step S23. If the 

15 controller control circuit 1 7 has received data from the controller 40 
to store in the RAM 174, the process proceeds to a step S24. At step 
S24, the bus control circuit 12 transfers the data of the controller 40 
stored in the RAM 174 to the W-RAM 14. The bus control circuit 
12, when completing the data transfer to the W-RAM 14, returns the 

20 process back to the step S21 to repeat execution of the step S21 - the 
step S24. 

The Figure 15 and Figure 16 flowcharts show the example 
wherein, after the bus control circuit 12 has transferred data from the 
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RAM 174 to the W-RAM 14, the CPU 1 1 processes the data stored in 
the W-RAM 14. However, the CPU 1 1 may directly process the data 
in the RAM 174 through the bus control circuit 12. 

Figure 17 is a flowchart for explaining the operation of the 
controller control circuit 17. At a step S3 1, it is determined whether 
there is data to be written from the bus control circuit 12. If there is 
not, the data transfer control circuit 171 waits until there is data to 
write-in from the bus control circuit 12. If there is data to be written, 
at a next step S32 the data transfer control circuit 171 causes the 
RAM 174 to store commands for the first to the fourth channels 
and/or data (hereinafter abbreviated as "command/data"). At a step 

533, the command/data for the first channel is transmitted to the 
controller 40 being connected to the connector 181. The control 
circuit 442 performs a predetermined operation based on the 
command/data to output data to be transmitted to the image 
processing apparatus 10. The content of the data will be described 
below in explaining the operation of the control circuit 442. At a step 

534, the data transfer control circuit 171 receives data output from 
the control circuit 442, to cause the RAM to store the data. 



At a step S3 5 the command/data for the second channel is 
transmitted to the controller 40, in a manner similar to the operation 
for the first channel at the steps S33 and S34. The control circuit 442 
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performs a predetermined operation based on this command/data to 
output the data to be transmitted to the image processing apparatus 
10. At a step S36 data transfer and write-in processes are carried out 
for the second channel. Meanwhile, at a step S37, the command/data 

5 for the fourth channel is transmitted to the controller 40. The control 
circuit 442 performs a predetermined operation based on this 
command/data to output the data to be transmitted to the image 
processing apparatus 10. At a step S38 data transfer and write-in 
processes are carried out for the third channel. Furthermore, at a step 

10 S3 9, the command/data for the fourth channel is transmitted to the 
controller 40. The control circuit 442 of the controller 40 performs a 
predetermined operation based on this command/data to output the 
data to be transmitted to the image processing apparatus 10. At a 
step S40 data transfer and write-in processes are carried out for the 

15 fourth channel. At a subsequent step S41, the data transfer circuit 
171 transfer in batch the data which have received at the steps S34, 
S3 6, S3 8 and S40 to the bus control circuit 12. 

In the above identified manner, the data for the first channel to 
the fourth channel, that is, the commands for the controllers 40 
20 connected to the connectors 1 8 1 - 1 84 and the operating state data to 
be read out of the controllers 40, are transferred by time-divisional 
processing between the data transfer control circuit 171 and the 
control circuit 442 respectively within the controllers 40. 
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Figure 18 is a flowchart explaining the operation of the 
controller circuit 44. First, at a step S51, it is determined whether or 
not a command has been input from the image processing circuit 10 
to the control circuit 442. If no command has been inputted, the 
controller circuit waits for a command. If a command is input, at a 
step S52 it is determined whether or not the command inputted to the 
control circuit 442 is a status request command (command "0"). If a 
command "0" is detected, the process proceeds to a step S53, wherein 
a status transmitting process is carried out. 

At the step S53, where the CPU 1 1 outputs the command "0", 
the data in the format as shown in Figure 13 is transmitted and 
received between the image processing apparatus 10 and the 
controller 40. On this occasion, the control circuit 442, when 
receiving the command "0" data configured by 1 byte (8 bits), 
transmits TYPE L (1 byte), TYPE H (1 byte) and the status. Here, 
TYPE L and TYPE H are data for identifying the function of a device 
or apparatus being connected to the joyport connector 46, which are 
inherently recorded in the RAM cartridge 50. This makes possible 
recognition by the image processing apparatus 10 as to what 
extension device (e.g., a RAM cartridge 50 or other extension devices 
such as a liquid crystal display) is connected to the controller 40. 
The status is data representative of whether or not an extension 
device such as a RAM cartridge 50 is connected to the port and 
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whether or not the connection of the extension device is after 
resetting. 

On the other hand, at the step S52 if the determination reveals 
that there is not a command "0", it is determined at a step S54 
whether or not the inputted command is a pad-data request command 
(command "1"). If it is a command "1", the process proceeds to a 
step S55 where the process of transmitting pad data is performed. 
Specifically, where the CPU 1 1 outputs a command "1", the data in 
format as shown in Figure 14 is transmitted and received between the 
image processing apparatus 10 and the controller 40. On this 
occasion, the control circuit 442, if receiving command "1" data 
configured by 1 byte (8 bits), transmits the data of 14 switches (16 
bits) of B, A, G, START, upper, lower, left, right, L, R, E, D, C and 
F; the data of JSRST (1 bit); and the data of the counter 444X and the 
counter 444Y (16 bits). By transmitting these data to the image 
processing apparatus 10, the image processing apparatus 10 
recognizes how the operator operated the controller 40. Thus, these 
data are utilized for modifying the image by the image processing 
apparatus 10 in accordance with the operating state of the controller 
40 as manipulated by the player. 



At the aforesaid step S54, if the determination reveals that 
there is not a command " 1 ", it is determined at step S56 whether or 
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not the input command is a read-out request command (command 
"2") for data associated with the RAM cartridge 50 to be connected to 
the extension connector. If it is a command "2", the process proceeds 
to a step S57 where the process reading out of the extension 
5 connector is performed. Specifically, where the CPU 1 1 outputs a 
command "2", the data in format as shown in Figure 13 is transmitted 
and received between the image processing apparatus 10 and the 
controller 40. On this occasion, when the control circuit 442 receives 
command "2" data configured by 1 byte (8 bits), address H 

10 representative of the higher-order bits (8 bits) of address, address L 
representative of the lower-order bits (3 bits) of address, and address 
CRC (5 bits) for checking for error in address data transmitted and 
received, the control circuit 442 transmits data stored in the RAM 
cartridge (32 bytes) and CRC (8 bits) for checking for data errors. In 

15 this manner, the connection of the RAM cartridge 50 (or other 

extension devices) and the image processing apparatus 10 enables the 
image processing apparatus 10 to process data from the RAM 
cartridge 50, etc. 

At the aforesaid step S56, if the determination is not a 
20 command "2", it is determined at a subsequent step S58 whether or 
not the inputted command is a write-in request command (command 
"3") for information associated with the RAM cartridge 50 being 
connected to the extension connector 46. Where it is the command 
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"3", the process of data read-out is carried out at a step 59 for the 
RAM cartridge 50 being connected to the extension connector 46. 
Specifically, if the CPU 1 1 outputs a command "3", the data shown in 
Figure 14 is transmitted and received, in response to the command 
5 "3", between the image processing apparatus 10 and the controller 40. 

That is, when the control circuit 442 receives command "3" 

O 

O data configured by 1 byte (8 bits), address H representative of the 

s:: 
I 

higher-order bits of address (8 bits), address L representative of the 
lower-order bits of address (3 bits), address CRC for checking for 
10 error in address data transmitted and received (5 bits), and data to be 
transmitted to the RAM cartridge 50 (32 bytes), it transmits CRC for 
checking for error for data received (8 bits). In this manner, the 
connection of the extension device 50 and the image processing 
apparatus 10 enables the image processing apparatus 10 to control the 
15 extension device 50. The connection of the extension device 50 and 
the image processing apparatus 10 also drastically improves the 
function of the controller 40. 

If at the aforesaid step S58 the determination is not a command 
"3", it is determined at a step 60 whether or not it is a reset command 
20 (command 255). Where it is the reset command (255), the process of 
resetting the counter 444 for the joystick 45 is performed at a step 
S61. 
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Where the CPU 1 1 outputs a reset command (command 255), 
the data shown in Figure 19 is transmitted and received between the 
image processing apparatus 10 and the controller 40. That is, the 
control circuit 442 of the controller 40, if receiving command 255 
data configured by 1 byte (8 bits), outputs a reset signal to reset the X 
counter 444X and the counter 444Y, and transmits aforesaid TYPE L 
(1 byte), TYPE H (1 byte) and the status. 

An explanation is now made concerning changing the camera 
perspective (point of eye) in a three-dimensional space. That is, 
where in the conventional 3D game there exists between a camera 
and an operable object (e.g., Mario) another object (e.g., a wall or an 
opponent character) as shown in Figure 20, the operable object or 
Mario cannot be viewed or "photographed" by the camera. In 
contrast, it is possible in the present invention to continuously 
display Mario at all times by turning the camera around Mario up to a 
lateral side thereof as shown in Figure 20. 

Stated briefly, where the objects are situated as shown in 
Figure 21, a determination is made of a collision with a topographical 
polygon extending from Mario's side, at several points on a straight 
line between Mario and camera. On this occasion, a check is made 
for a polygon that is perpendicular to an XZ plane inside a radius R 
from each point. The process of turning-around of the camera is 



performed on a polygon P determined as collisional. The wall 
surface P is expressed by the flat-plane equation as given by Equation 

(1). 

Ax + By + Cz + D=0 ... (1) 

5 The correction in camera position is done by moving the 

"camera" in parallel with this plane P. Incidentally, the angle of Y- 
axis in parallel with the plane is calculated by the flat-plane equation. 

Explaining in further detail in conjunction with the Figure 22 
flowchart, the No. n of a polygon to be collision-determined is 
10 initialized (n=l) at a first step S101. At a next step S102, it is 

determined whether or not the number N of polygons to be checked 
for and the polygon No. are equal, that is, whether or not a collision- 
determination has been made at a next step SI 03. 

Figure 23 shows in detail step S103, i.e., an illustrative 
15 collision-determination routine. Before explaining this collision- 
determination routine, reference is made to Figure 24 and Figure 25 
which show wall data to be collision-determined. That is, the wall 
data is depicted as in Figure 24 wherein triangular polygons as in 
Figure 25 are gathered together. These respective polygons are 
20 stored as listing of wall polygons in a memory. 
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At a first step S201 in Figure 23, a point Q (Xg, Yg, Zg) and a 
radius R are input. The point Q is a point to be checked and the 
radius R is a distance considered to be collisional against the wall. 
At a next step S202, a wall-impingement flag is reset. At a step 
5 S203, it is determined whether or not the wall-polygon list explained 
hereinbefore is stored in the memory. If there exists a wall polygon 
list, it is determined at a next step 204 whether or not the same 

l""! 

% polygon is a polygon to be processed by turning around of the 

j'h 

U camera. At this step S204, If so, the process proceeds to a step S205. 

\i 

m 

;; =! io At the step S205, the distance (dR) between the point Q and 

H- the plane of wall polygon is calculated according to Equation (2). 

H' 

.S3!. 

nj dR = AXg + BYg + CZg + D ... (2) 

Then at a step S206 it is determined whether or not the 
distance dR calculated at the step S205 is smaller than the radius R. 
15 When the distance dR is greater than the radius R, there occurs no 
collision between the Mario and the wall, and accordingly the process 
returns back to the aforesaid step S203. . 

If "Yes" is determined at the step S206, that is, when I dR I < R, 

a calculation is made at a step S207 according to Equation (3) for 
20 determining positional coordinate (Xg', Yg', Zg') of a point of 
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intersection Q' between a straight line extending from the point Q 
vertically to the wall polygon P and the plane of the wall polygon. 

Xg' = Xg + A x dR 
Yg' = Yg + B x dR 
Zg' = Zg + CxdR ... (3) 

Then at a next step S208, it is determined whether to not the 
point Q' is on the inner side of the polygon (within the range). 

At step S208, it is determined onto which plane projection is to 
be made in dependence upon the direction of the wall (a value A). 
That is, when A < -0.707 or A > 0.707, projection is onto a YZ plane 
shown in Fig. 26. Otherwise, projection is onto an XY plane. Where 
the projection is onto the YZ plane, it is determined whether or not in 
Figure 27 the point Q' is on an inner side of the polygon PL 

Meanwhile, where projection is onto the XY plane, it is 
determined on the point Q' and apexes of the polygon PI in Figure 28 
whether the value of counterclockwise cross product is positive or 
negative. That is, when C in the polygon-plane equation is C 2: 0, if 
each of the resulting cross products is 0 or negative, then 
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determination is that the point Q' is on the inner side of the polygon 
P. 

(Yl -Yq)x(X2-Xl)-(Xl -Xq)x(Y2 -Yl) < 0 

(Y2 - Yq) x (X3 - X2) - (X2 -Xq) x (Y3 -Y2) <; 0 

(Y3 - Yq) x (XI - X3) - (X3 -Xq) x (Yl -Y3) <; 0 ... (4) 

Meanwhile, when C < 0, if each of the resulting cross products 
is 0 or positive, then determination is that the point Q' is on the inner 
side of the polygon P. 

(Yl - Yq) x (X2 - XI) - (XI -Xq) x (Y2 -Yl) > 0 

(Y2 - Yq) x (X3 - X2) - (X2 -Xq) x (Y3 -Y2) > 0 

(Y3 - Yq) x (XI - X3) - (X3 -Xq) x (Yl -Y3) * 0 ... (5) 

In this manner the point Q' is checked at the step S208 whether 
it is on the inner side, of the polygon or not, and at a step 209 it is 
determined whether or not the point Q' is on the inner side of the 
polygon. If "Yes" at this step S209, the wall-impingement flag that 
had been reset at the aforesaid step S202 is set (step S210). 
Thereafter the process returns to Figure 22. 



Note that the abovestated collision-determination is merely one 
example, and it should be recognized that the collision-determination 
is possible by other methods. 

Referring back to Figure 22, after the collision-determination 
at the step S 1 03 , it is determined at a step S 1 04 whether or not a wall 
-impingement flag is set. If "No" at this step S 1 04, the process of 
turning around is unnecessary, so that the No. n of a point to be 
checked is incremented at step S 105 and the process returns back to 
the step SI 02. 

If "Yes" at the step SI 04, it is determined at step SI 06 and step 
SI 07 whether it is on a back side of the wall. That is, the 
directionality of the polygon is determined. Whether the polygon is 
directed to the camera (the point of view) or not can be determined 
by examining the sign of the dot product of a normal vector N and an 
eye (point of view) vector V in Figure 29. The conditional 
expression therefore is given by Equation (6). 

A = V-N = VxNx + VyNy + VzNz (6) 

With Equation (6), determinations are respectively possible 
such that if A :> 0 the wall is directed to the camera (frontward) while 
if A < 0 the wall is directed to a backside of the wall. If a plane 
existing between the camera and Mario is directed frontward relative 
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to the camera, the turning-around of camera in Fig. 30 is not done. In 
this case, the No. n of the point is incremented at a step SI 05, and the 
process returns back to the step SI 02. 

If the plane between the camera and Mario is directed 
5 backward, the answer to the step S 1 07 becomes "Yes", and the 
turning-around process is carried out at subsequent steps S108 and 
S 109. At the step S 1 08, the angle of movement through which the 
position of camera (photographing position) is altered based on the 
flat-plane equation for the wall. That is, the flat-plane equation in 
10 terms of three points PI (XI, Yl, Zl), P2(X2, Y2, Z2), P3 (X3, Y3, 
Z3) on the flat-plane equation is expressed by a multi-term equation 
of Equation (7). 

Ax + By + Cz + D = 0 

where, A = Y1(Z2-Z3) + Y2(Z3-Z1) + Y3(Z1-Z2) 

15 ■ B = Z1(X2-X3) + Z2(X3-X1) + Z3(X1-X2) 

C = X1(Y2-Y3) + X2(Y3-Y1) + X3(Y1-Y2) 

D = X1(Y2Z3-Z2Y3) + Y1(Z2X3-X2Z3) + 
Z1(X2Y3-Y2X3) ... (7) 
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The angle Ry of the normal vector with respect to the Y- 
axis is given by Equation (8). 

Ry = tan" 1 (A/C) ... (8) 

Therefore, the tuming-around angle of camera is either 
Ry + 90° or Ry - 90°. That is, at the step SI 09 the camera is 
rotationally moved about Mario, or the operable object, in 
either direction Ry + 90° or Ry - 90°. Specifically, the 
movement is to a location closer to the presently-situated 
camera position (C in Figure 21). 

Although the present invention has been described and 
illustrated in detail, it is clearly understood that the same is by 
way of illustration and example only and is not to be taken by 
way of limitation, the spirit and scope of the present invention 
being limited only by the terms of the appended claims. 



